Domain Expertise vs. Validation Expertise: Essentials of Validating Intelligent Systems

The present short paper deals with fundamental problems in Computer Science, in general, and in Artiicial Intelligence (AI), especially. It addresses the question how to deal with the problem whether or not complex interactive systems really do what they are expected to do. In fact, the focus of the presentation of this paper is even more narrow: The essential reasons why validation expertise usually implies domain expertise. More technically, this is a contribution to the research and development area of intelligent systems validation. As those systems are typically interactive and, thus, utilized in open loops of human{machine interactions, the problem of their validity is substantially complicated. A certain family of validation scenarios is adopted. Within this framework, we ask for the power and the limitations of these validation approaches. The expertise necessary and suucient to accomplish successful validation is of some particular interest. One of the key question is for the comparison of domain expertise and validation expertise, for instance. The goal of the paper is to communicate just one key insight. The area of inductive inference of recursive functions is taken as a case for complex interactive systems validation. Interactive scenarios of validating inductive inference algorithms are formalized. Computability theory is providing a rich source of theoretical concepts and results suitable for the focused investigations. This background is mostly skipped. These concepts are exploited for characterizing the expertise necessary and suucient in the validation of inductive inference systems. Particular emphasis is put on relating validation expertise and domain expertise by means of relativized computability concepts. One of the key results on validation of inductive learning systems exhibits that validation expertise necessarily implies the expertise for solving the focused learning problems. The disciplinary results are interpreted from the general perspective of complex interactive systems validation. This paper is somehow special, because it combines the transparencies of the author's talk (to be) delivered at the \1997 Annual Symposium of the Meme Media Laboratory" (held on March 9, 1998) with the text of a short research paper. From the one perspective, it is some paper illustrated by presentation slides. From the other perspective, it is, perhaps, only a collection of annotated transparencies. 1 Background The present paper and, in particular, the presentation is based on common work with my colleagues and friends 1 Gunter Grieser and Steeen Lange.